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Separation of Close Eigenvalues af a Real Symmetric 

Matrix' 

J. B. Rosser, C. Lanczos, M. R. Hestenes, and W. Karush 

In order to test two methods, one proposed by ('. Lanczos and the other by M. 1^, 
Hestenes and W. Karush, for the numerical calculation of eigenvalues of symmetric matrices, 
an 8 by 8 matrix is constructed that has several sets of eigenvalues close together. The 
application of the two methods to this test matrix is described, and in addition, a third method 
for dealing with such problems is proposed. 



In recent publications ^ ^ two methods have been 
proposed for finding eig'envalues of real symmetric 
matrices. In order to make a numerical comparison 
between the metliods, an 8 by 8 matrix was especially 
designed (see appendix 1) and th(^ two methods were 
used independently to get all eight eigenvalues and 
eigenvectors of the matrix. In order that the test 
be a severe one, the matrix was designed with several 
sets of eigenvalues very close together. In order to 
separate these eigenvalues, special modifications of 
the two methods were devel()j)ed for the separation 
of close eigenvalues (see appendixes 2 and 3). 

The method of Lanczos (see footnote 2 and ap- 
pendix 2) seems best adaptecl for use by a hand com- 
puter using a desk computing machine. In the 
present case, the computation according to Lanczos' 
method was carried out by a hand computer, and 
required of the order of 100 hours computing time. 

The method of Hestenes and Karush (see footnote 
3 and appendix 3) seems best adapted for use by 
machine ('omj)utation. in the present case, the 
computation according to the method of Hestenes 
and Karush was carried out on an IBM Card -Pro- 
gramed Electronic Calcidator. Considerable time 
was spent })y Karush in becoming familiar with the 
machine^ so that it is difficult to say just how long 
the computation would require of an experienced 
operator. Probably 3 or 4 days would be ample. 

During and since the computations described 
above, there has been much discussion of the problem 
of separating close eigenvalues of a real symmetric 
matrix. Besides the methods offered in appendices 2 
and 3, we wish to offer the following modifications of 
the familiar power method. 

First let us consider the case where only the 
numerically largest eigenvalue, Xi, and the corre- 
sponding eigenvector, Vi,oi a matrix -A are desired. 
We may assume Xi to be positive, since otherwise we 
treat —A. 

Suppose Xi, . . . , Xn are the eigenvalues of A 
in decreasing order, and Vi, . . . , Vn are the corre- 
sponding eigenvectors. If no other eigenvalue is near 
Xi, one can find Xi and Vi by the standard power 
method. In order to be able to compare the modi- 
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fications for the case where anotlnn* cngen value is 
near Xi, we review the power method. 

First, one chooses a vector x. This has a represen- 
tation 

X = CiVi + C2V2+ . . . +CnVnj 

but as yet we do not know the c's or v's. By itera- 
tions of the step of operating on a vector with a 
matrix, we form Ax, A^x, A^x, .... 'J'he represen- 
tation of A^x is 

A''x = C,KVi + C2\U2+ . . .+CnKVn^ 

If C]9^0 (which is the case except in very extra- 
ordinary circumstances), then for sufficic^ntly large 
A', CiXj"' will be much greater than CtXf(i>l), 
since Xi>|Xi| (i>l). Thus A^x is nearly a multiple 
of Vi. By normalizing in the desired fashion, an 
approximation (of any desired degree of accuracy) 
for Vi is obtained, from which an approximation to 
Xi can be obtained. 

In case X2=Xi, any Unear combination of ^i and Vo 
will serve perfectly well as an eigenvector corre- 
sponding to Xi. The power method just outlined 
will yield a linear combination of Vi and r. in such a 
case, and so no difficulty arises. 

Suppose X2 is nearly as great as Xi, but all other X's 
are appreciably smaller. Then one will have to take 
A^ excessively large before C2X^ is small compared to 
Ci\i . Two possible procedures for curtailing the 
labor are as follows. 

In the first, we take A^ large enough so that CtX^- is 
small compared to CiXf or C2><2 for />2. Then 
approximately, 

A^X = C,\'^VI + C2KV2. 

Choose two vectors y and z. Put 

where (u,v) denotes the inner product of the vectors 
u and V. Then Xi and X2 are the two roots of the 
quadratic equation 
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To prove this, write 
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Then clearly the determinant is zero whenever 
X=X2 or X==X2, so that X] and X2 are roots of (1). 

In exceptional cases, the coefficients of (1) are all 
zero. This can happen if Xi==^X2, or Ci = 0, or C2=0, 
or in case the projections of y and 2 are not independ- 
ent on the subspace spanned by Vi and V2 (this is the 
case where 



b 
d 



= 0, 



and can generally be treated by mereh^ choosing a 
different y and z). 

The case where Xi, X2, and X3 are all nearh^ equal 
but the remaining X's are small can be handled 
similarl}^, and leads to a third-degree equation defined 
b}' a fourth -order determinant. 

Returning to the case where Xi and X2 are nearly 
equal, and other X's are smaller, an alternative pro- 
cedure makes use of Chebyshev polynomials. Sup- 
pose that a sufficiently high value of A^ has been 
used in order to establish that there are one or more 
roots in the vicinity of some value ju (which is ap- 
proximately Xi, and hence also approximately X2, 
since Xi and X2 are nearly equal), and that the other 
roots are appreciably less than ix in absolute value. 
In particular, — ju is a lower bound for the roots. 
Now instead of taking powers of A, we take powers 
of a polynomial in A, noting that 



{P{A)Yx-- 



--c,{P{\,)Yv,+ 



+ c,(F(X,r^, 



If now we choose P(X) so that P(Xi) and P(X2) are 
near 1, and P(X) has a large slope in the neighbor- 
hood of Xi and X2, then P(Xi) and P(X2) will have a 
ratio appreciably less than X1/X2, and hence powers 
of P{A) will eliminate V2 relative to Vi faster than 
powers of A. 

We first apply ^4 enough times to eliminate all ?;'s 
except i\ and V2, and then apply P{A). In order to 
insure that P{A) does not bring back the y's already 
eliminated, it suffices that |P(X)|<1 for— /x<X</x. 
To do this and simlutaneously maximize the slope 
of P(X) at X=At for P(X) a polynomial of degree M, 
it suffices to take 



P{\) = TM(^y 



where Tm is the Chebyshev polynomial of degree M.* 
Actually, it may be more efficient to use different 
polynomials at different stages in the proceedings. 
The optimum choices of the polynomials will depend 
on the distribution of the X's, naturally. As this is 
not known ahead of time in a given case, one must 
depend on a combination of experience and alert 
improvisation to get a good choice of polynomials. 
We now turn to the case where one wishes to find 
all eigenvalues and eigenvectors. If any sort of fast 
computing machinery is available, one can probably 
proceed best by a combination of the power method 
plus orthogonalization on the eigenvectors already 
known. In particular, suppose Xi and Vi are known. 
We can start with x and orthogonalize it with respect 
to Vi. That is, we replace x by 

For the resulting vector, we have Ci=0. Hence, if 
we apply powers of A to it, we get the eigenvector 
corresponding to the eigenvalue next greatest after 
Xi in absolute value. Unfortunately, since we do not 
know Vi exactly, we cannot in general determine x 
to be exactly orthogonal to Vi, and so cannot insure 
Ci = 0. We thus face the possibility that CiXf ma}^ 
again be large. If, however, we orthogonalize with 
respect to Vi from time to time, we repeatedly cut 
down the size of CiXf . On a fast machine, orthog- 
onalization is a quick procedure, and it is probably 
worthwhile to alternate the steps of orthogonalization 
and operating with A, 

If Xi and X2 and Vi and V2 are known, one orthogonal- 
izes with respect to both Vi and V2 between each time 
that one operates with A. 

If at any point in the procedure, one encounters 
two close eigenvalues, one is trying to find the largest 
unknown eigenvalue, and so can apply the methods 
noted above (which are not disturbed b^^ the frequent 
orthogonalizations). However, now that one plans 
to find all eigenvalues, alternative quicker methods 
are available for separation of close eigenvalues, 
depending upon knowing all other eigenvalues and 
eigenvectors. For example, suppose A has eigen- 
values 1, 2, 2.95, 3.05, 4, and 5. Successively getting 
the largest eigenvalue twice by the power method 
plus orthogonalization, we readily get the eigen- 
values 5 and 4, and their eigenvectors. We now 
discover that there are troubles in the neighborhood 
of 3. Essentially, we ''postpone" treatment of this 
point bv putting B='^I—A. Then the eigenvalues 
1, 2, 2.*95, 3.05 of A lead to the eigenvalues 2, 1, 
0.05, —0.05 of B. Going now for the largest eigen- 
values of B, we quickly get 2 and 1. We now have 
all eigenvalues and eigenvectors of A except 2.95 
and 3.05 and their eigenvectors. Also we now know 
that there are just two remaining eigenvalues, and 
that both are near 3. We now consider C=A—2.^L 
This has eigenvalues 0.05 and 0.15, and the power 

* G. Polya and G. Szego, Aufgaben und Lehrsiitze aus der Analysis 11, p. 91 
(Dover Publications, New York, N. Y., 1945). 
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method plus orthogonalization qiiickl}^ gives the 
larger of these. 

This method will run into difReulties if there are 
two pairs of close eigenvalues. An alternative pro- 
cedure that will take care even of this case is the 
following. Suppose we have eigenvalues Xi = 5, 
X2=4.05, X3=3.95, X4=3, X5==2.05, X6-1.95, and 
X7=l. We quickly find Xi and Vi. Trying for X2, we 
find trouble. By starting with some x and alter- 
nately orthogonalizing with respect to Vi and oper- 
ating with A, we keep Vi out, and eventually elimi- 
nate Vi, v^, Vq, and v^. We now have a certain linear 
combination of V2 and Vs, which we may as well 
call U2. We now repeat the procedure, except for 
starting wdth a different x. We then get a Us that 
is also a linear combination of V2 and v^. Except in 
the most extraordinarily unfortunate cases, U3 will 
be independent of U2. One can insure this inde- 
pendence by orthogonalizing with respect to U2 
throughout the computation of 1/3- However, it is 
scarcely worth while, except perhaps in the choice of 
the initial x. 

Since U2 and Us are independent linear combina- 
tions of V2 and ^3, it follows that a vector is orthogonal 
to both of V2 and ^3 if and only if it is orthogonal to 
both of U2 and u-s. To find X4 and v^, we would wish 
to orthogonalize with respect to all of Vi, V2, ih- We 
can get the same effect if we instead orthogonalize 
with respect to Vi, U2, and u^ (this is most conven- 
iently done if 1/3 is taken orthogonal to 1^2)- Thus 
we can now proceed to get X4 and tu, although we do 
not yet know X2, X3, V2, or v^. We again encounter 
difficulty with v^, and Vq because X5 and Xe are near 
together. However, we can get a U5 and a u^, which 
will suffice to let us obtain X7 and Vj. Now, by 
orthogonaHzing with respect to Vi, V4, U5, Uq, and V7, 
we can readily separate X2 and X3 by working with 
powers of ^—3.9/. Then we get X5 and \ by work- 
ing with powers of ^—1.9/. 

Appendix 1. Construction of a Test Matrix 

In order to get eigenvalues very close together without 
using many significant digits in the coefficients, it seemed 
necessary to use irrational numbers. Accordingly, a search 
was made for 2 by 2 symmetric matrices with eigenvalues, 
some of w^hich were near together. We decided on the fol- 
lowing four, where the eigenvalues are written to the right 
of the matrices: 
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The approximate numerical values of these oight eigenvalues, 
written in descending order are: 

102. 005 
102. 000 
101. 990 
100. 000 
100. 000 

0. 010 

0. 000 
- 102. 005 

One eigenvalue, namely 100, is e.xactly repeated. These 
2 by 2 matrices were then mixed together into an 8 by 8 
matrix as follows: 
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The two 4 by 4 matrices occurring in the corners of P were made by a scheme due to Sylvester, ^ with the result that P 
has the property 

pTp=lOI 

(we use P^ to denote the transpose of P). We then defined A to be P^B P. The matrix A is then 
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•5 T. Muir, History of determinants III, 289 (Macmilbn and Co., Ltd., London, 1920). 
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with the eigenvalues and eigenvectors 
X,= lOV 10405= 1020.04901843 



vi={2, 1, 1, 2, 102-VTO4O5, 102-V10405, - 204 + 2vl0405, - 204+ 2V 10405) 

= {2, 1, 1, 2, -0.004901843, -0.004901843, 0.009803686, 0.009803686) 
X2=1020 

t;2=(l, -2, -2, 1, 2, -2, 1, -1) 
X3=510+100V26= 1019.90195136 
«;3=(2, -1, 1, -2, V26, -5 -V26, -10 -2V26, 10+2V26) 

= (2, -1, 1, -2, 10.09901951, -10.09901951, -20.19803903, 20.19803903) 
X4=X5=1000 

t;4=(l, -2, -2, 1, -2, 2, -1, 1) 
v^=(7, 14, -14, -7, -2, -2, -1, -1) 
X6 = 510- 100V26= 0.09804864072 
v,= {2, -1, 1, -2, 5-V26, -5 + V26, -10+2V26, 10-2V26) 

= (2, -1, 1, -2, -0.099019514, 0.099019514, 0.198039027, -0.198039027) 
X7 = 

Vt=(1, 2, -2, -1, 14, 14, 7, 7) 
Xg= - lOV 10405= - 1020.04901843 
y«=(2, 1, 1, 2, 102 + v'T0405, 1024-Vl0405, -204-2Vl0405, - 204- 2V 10405) 

= {2, 1, 1, 2, 204.0049018, 204.0049018, -408.0098037, -408.0098037). 



Appendix 2. Determination of the Char- 
acteristic Roots in the Method of Lanczos 

The method of minimized iterations (cf. footnote 2) leads 
to the construction of a successive set of orthogonal vectors 



ho, hh 



.,6„ 



(1) 



starting with the trial vector 60. Each iteration is associated 
with two scalars ui and jSj; they become the pivotal elements 
of the eigenvalue problem. 

If the vectors (1) are introduced as an auxiliary reference 
system, the original matrix A is transformed into the following 
^'codiagonal" form (omitting the zero elements):^ 



«o 
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1 

«2 



Pn- 



Oin-iy 



(2) 



The solution of the principal axis problem requires the 
construction of a set of polynomials Pi{x) on the basis of the 
recurrence relations 



Pi+l{x) = {x~ai)pi(x)-^iPi-l{x) 



(3) 



'" The /3i of the present report corresponds to the fii-i of the reference cited in 
footnote 2. 



starting with 

Po{x) = 1 

pi(x)=x — aQ 

and ending with pn{x). The roots of the algebraic equation 

Pn{x) = (4) 

yield the n eigenvalues 

a:=Xi, X2, . . ., X„. (5) 

The matrix (2) is not symmetric because the vectors hi are 
not normalized in length. In order to normalize hi and thus 
symmetrize the matrix C, we introduce the quantities 



Yi=^/Fi- 



(6) 



If the original matrix A is symmetric, then the /3^ are all 
positive and the ji all real. The sign of the 7^ shall be taken 
as positive. 

The normfactors 

a-i = V6f (7) 

are now expressible in terms of the 7,;. Assuming that the 
original trial vector bo was chosen of the length 1 — that is, 
a;o=l — we obtain 

Wi=7i72 . . . 7t. (8) 

The matrix A, if analyzed in the reference system^'of the 
normalized 



6=^ 



(9) 
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appears in the following symmetric form: 



71 ai 72 

72 «2 73 



7n-l 



(10) 



CLn-\J 



The quantities «» and ^i, obtained by the method of mini- 
mized iterations, contain the entire sohition of the eigenvalue 
problem. The eigenvalues are contained in the solution of 
the algebraic equation (4), while the components of the 
eigenvector u^, analyzed in the ftj-system, become: 



??o(Xi),?>i(Xz), . . . ,?>n-l(At). 

If we construct the matrix 

'^Po(Xl),Pl(Xl), . . . ,Pn-l(Xl)^ 



(11) 



Lpo(Xn),Pl(Xn), . . . , Pn-l(Xn). 



(12) 



then the matrix {)roduct 



U = PB. 



(13) 



> where B is the matrix of the hi, gives the matrix of the eigen- 
vectors Uij associated with the original matrix A. 

The orthogonality of the eigenvectors //, finds expression 
in the following relation: 



^l PlP2 . . . Pa 



ii^j) 



(14) 



In the given test-matrix a preliminary investigation of the 
matrix revealed that the largest eigenvalue is of the order of 
magnitude ± 1000. Hence all the elements of A were divided 
by 1000, thus obtaining a new matrix 



A,-- 



A 
1000 ' 



whose largest eigenvalue was of the order ± 1 . 
The trial vector 6o was chosen to be 



0, 1, 0, 0, 



.,0 



Then the method of minimized iterations was applied, obtain- 
ing the B matrix by putting the components of the vectors 
&o, ^1, ^2, . . ., 6n-iin successive rows. Each one of these vectors 
was corrected during the process of generation to become 
strictly orthogonal to the previous vectors. Hence 6„ must 
come out as identically zero, in spite of roundiiig errors. 
The associated tti and /Si, together with the 7i= V/^i, are tabu- 
lated as follows : 
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2 
3 
4 
5 
6 
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0.899 

. 1086629633 
. 7859177671 
-. 7935214279 
. 003963315517 
1. 0160663075 
1.0199110708 
1. 0000000030 






0. 096939 
. 039517948848 
. 4088977136 
. 0520498144977 
. 004021099703 
.081070421101 
. 0> 07048359779 


0.3113502850 
.03085117315 
.6394511034 
.001431717325 
. 06341214161 
. 0^3271729055 
. 058395451018 



We will now discuss the problem of obtaining the roots of 
the algebraic equation (4). Our procedure will be to obtain 
a good first approximation and then improve this approxima- 
tion to the full accuracy obtainable by 10 digit calculations.^ 

The separation of nearly equal roots is frequently a rather 
cumbersome task. In the present method the existence of 
nearly equal roots is an asset rather than a liability. The 
orthogonality relation (14) shows that exactly equal or nearly 
equal roots are only possible under singular conditions. If 
none of the ^k are small, then Xi and Xy cannot be essentially 
equal since a sum of all positive terms cannot vanish. If, on 
the other hand, a certain /S^ is zero or very small, this means 
that the polynomial Pn{x) separates into the product of two 
independent polynomials of low^er order, wiiich greatly sim- 
plifies the evaluation of the roots. 

The given numerical example is well adapted to demon- 
strate the behavior of equal or nearly equal roots. Since 
nearly equal roots operate as practically one root in the suc- 
cessive reduction of the trial vector 6o, we will obtain a very 
small hm already after m steps, where m is the number of es- 
sentially different roots. In the present problem we have 
only three essentially different roots. Owing to an accidental 
degeneracy, only tioo of these roots were strongly represented 
in ?>o- Hence ^2 is already small. The remaining vector 
again contained essentially but two roots, and thus ^^ is again 
small. Furthermore, we notice that /Se is very small and P^ 
almost negligible. 

Indeed, the fact that two roots of the given problem coincide 
has the consequence that feo should get reduced to zero in 
already seven steps, thus making /Jy exactly zero. That ^7 
is not exactly zero, but only to 9 decimal places, is due to 
rounding errors. 

The associated a? should give the double root X=l. Ac- 
tually 

«7= 1.0000000030 

is a very close approximation of the exact root. 

We also notice that already /So is very small, although 15 
times larger than ^i The associated 

««= 1.01091 10708 

is a close approximation of another of the nearly equal roots, 
namely 

X= 1.0199019514. 

These specific properties of the /3i have the consequence 
that the original equation of 8th order separates into equations 
of the order 

2 + 24-2+1 + 1. 

This yields eight approximate roots of our problem, without 
solving equations of higher than second order. 

The question is now, how^ to improve the accuracy of these 
approximations. The application of Newton's method for 
correcting a root is here out of question since w^e cannot con- 
struct the actual polynomial p^ix) without such rounding 
errors, which completely annihilate the desired accuracy. 
We can construct the successive polynomials Pi(X) for any 
given X by the recurrence relations (2), but for arbitrary X 
the coefficients of the final polynomial are marred by intol- 
erably large errors. 

The following perturbation method has quick convergence 
and operates numerically very satisfactorily. Let us assume 
that we possess a vector y, which approximates the solution 
of the eigenvalue problem. 



Then 



Ay — \y = Q. 
_yAy 



X=- 



y. 



(15) 

(16) 



is a very satisfactory approximation of a certain X^, because 
an error of first order in y causes an error of only second order 
in X. 



7 This means 10 decimal places if the largest eigenvalue is normalized to 1. 
The absolutely smallest eigenvalue may be zero or arbitrarily near to zero. This 
zero, however, cannot be ascertained to more than 10 decimal places. The 
relative accuracy of the smallest eigenvalue may thus become arbitrarily bad. 
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We will apply this principle to our problem in the following 
sense. Let us assume that X is an approximate root of the 
polynomial PnW- We now construct the components of 
the vector y by evaluating the following recurrent sequence: 

2/0=1 

_(X— ao)2/o 



2/1 = 



2/2 = 



{\-ai).yi-yiyo 

72 



The vector 



_ (X — a„-2)2/n-2 — Tn-22/n-3 

n-1 — 

7n-l 

2/n=(X — a„-l)2/„-l — Tn-l?/n-2. 



(17) 



y=(yo,yh • • • » Vn-i) (18) 

taken in the reference system of the h-, satisfies the equation 

Ay-'\y = 

in all its components except the last one, where we get 

(Ay—\y)n-i=—yn. 
Hence * 

yAy=\y^ — ynyn-i. 
Substitution in (16) gives 



k = i on: 



X = X- 



^yl 



Actually it is entirely accidental that the equation where 
the error occurs shall be the last one. We can start our 
recurrences from both ends of the matrix and join the two 
sets at an arbitrary point i. The error will then occur in 
the ith rather than the last equation. 

Now the correction of the error of X will be most effective 
if the error of the equation (15) appears in that particular 
component i that is associated with the absolutely largest pi. 
We designate this particular yi and add to the sequence (17) 
another sequence that starts from the other end and proceeds 
in opposite direction : 

2/:-i=i 

, _ (\-an-i)yn-l 
f/n-2 — 

Tn-1 

(X — a„-2)y^-2 — 7n-l ?/l-l 

2/n-3 — 

7n-2 



,_^^ C\-ai)y'i — yi+iyi+i 

7i 



(19) 



W^e adjust this sequence to the sequence (17) by multiplying 
every component by 

(20) 



P=^- 
Vi 



We now construct our vector y by choosing its components 
from the yk series up to k = i, and from the y'k series from 



yo=yQ,yi^y,- - ',yi=yr = py'i) 
yi+i=pyUh- • ',Vn-i==pyn-i ) 

The error occurs in the ith equation and we obtain 

. T ■ yi(yi-i—pyi-i)yi 



(21) 



(22) 



The entire process can now be repeated, by replacing X 
by the new X. This process had such good convergence that 
after two steps the error was already pushed out beyond the 
10th decimal place. The entire set of Xi was thus obtained 
with relatively little difficulty and without involved cal- 
culations. 

After obtaining the X^, the P matrix was obtained by re- 
cursions. Finally, the product PB gave the matrix II of 
the eigenvectors Ui. This matrix was then normalized by 
dividing each row by the square root of the sum of the squares 
of the elements of each row. 

The resultant normalized matrix U' was now^ tested for 
orthogonality and for its eigenvector prop^ty. Under 
exact conditions we should get 



u- u!c = 

u'i AUk = 



(i^k) 
(i^k) 



(23) 



Actually, in view of the rounding errors, we do not get zero 
on the right side but two symmetric matrices 

Pik=Pki (24) 

and 

<^ik=<^ki (25) 

composed of small elements. We use these p^ and aik quan- 
tities to correct our solution. We evaluate 



(26) 



In view^ of the extreme closeness of some of the eigenvalues, 
the denominator of (26) becomes small for some i, k, and the 
corresponding €ik not negligible. We now form the matrix E^ 
composed of the non-negligible elements eik, wiiile the diagonal 
elements and the negligible €ik are replaced by zero. The 
corrected W matrix becomes: 



U'=U' + EU' 



(27) 



The rows of the corrected matrix give us the proper eigen- 
vectors with an accuracy of six decimal places. The Xj, I 
evaluated from these vectors, agreed with the previous X^ to ' 
10 decimal places. Comparison with the known exact values 
showed that all the 10 decimal places came out correctly for 
each one of the roots. ^ 

Appendix 3. Modification of the Method of ^ 
Hestenes and Karush 

The method of fixed a (see footnote 3) was used in the 
calculation. This consists in passing from one approxima- 
tion X for a characteristic vector to the next approximation x' 
by means of the formula 



w^here 



x' = x-{-oi^{x) 



^{x) = Ax—fjL{x)x, ix{x)-- 



{x,Ax) 
{x,x) 



Here a is a positive constant (independent of x) of the form 



0<^<l, 



5 The entire numerical work was carried out by Miss Fannie M. Gordon. 
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where /3 is optimally near 1. The above formula for x' is 
used to obtain convergence to an eigenvector Vi belonging to 
Xi, the largest (algebraically) eigenvalue. For an eigen- 
vector Vn belonging to Xn, the smallest eigenvalue, the formula 

x'=x—a^{x) 

is to be used. In practice a is at the disposal of the computer, 
and he may change its value at different stages of the calcula- 
tion according to his disposition and insight. The changes 
are easily made by having at hand several punched cards 
carrying different values of « and replacing at any time the 
single card in the deck carrying the value of a by another 
suc>^ card. 

The fixed a method is closely related to the power method. 
To illustrate this, suppose we are computing the least value 
Xs. After a certain number of steps the value of fiix) will 
be essentially constant from step to step, this constant repre- 
senting our computed value of Xs. Continued calculation 
leads to improvement in the vector x. The iteration fornuila 
with ^{x) replaced by its expression in terms of Ax and 
X becomes 



.' = «{(i + M)/-.l 



with / the identity matrix. That is, approximately, 

x/ = a{{fi'\i-\-il-^')\,)I-A]x, 

where |8' = 1//3 is near I . Thus, except for a normaHzation 
factor a, this is the power method applied to \[I — A with 
x; near Xi. In essence we have shifted the origin close to 
Xi, thereby making X,s the dominating eigenvalue in absolute 
value. The normalizing factor a guarantees that the lengths 
|a;i will increase and converge. 

The above procedure was used to calculate all eigenvalues 
and eigenvectors by the technique of orthogonalizing to 
eigenvectors already known, in the manner described in the 
text. As more eigenvectors are ol)tained the j)aranieter a 
is allowed to assume a greater value, this value in each new 
case being of the form /3/A/, where M is the spread of the 
eigenvalues for the subspace in question. Thus if Xs and vs 
are known, the iteration operates in the 7-dimensional sub- 
space orthogonal to v^, where the appropriate value of M 
is Xi — X7. 

Multiple roots offer no difficulty. Thus in the case of 
X4 = X5=10O0, the iteration first leads to the eigenvalue 
X4=1000 and to some corresi)onding eigenvectors v^. Run- 
ning orthogonal to V4 (and other known eigenvectors) we 
obtain Xs^lOOO and the eigenvector y.r, orthogonal to v^. 

Close roots may be treated as follows. At first the close 
roots are ignored and because Xi, X2, X3 are nearly equal and 
Xe, X7 are nearly equal one obtains by the orthogonalization 
technique eight independent vectors 



instead of the true eigenvectors 

VuV2yl%V4,V5,l%V7,Vs. 

Here u\, U2, ih are linear combinations of Vi, v-zy v-s and u^j Ui 
are linear combinations of vq, v-j (see text). To find the first 
vector Vi we apply our iteration procedure in tlu^ 3-space 
spanned by these vectors. That is, wv run orthogonal to 
Va, v^, u^, u-i, Vs and use a (large) a appropriate to the 3-space. 
Having obtained Vi we run orthogonal to Vi to obtain V2. To 
obtain v^ we do not require the a iteration method; we need 
only orthogonalize to Vi and V2 in the 3-space. Notice that 
this procedure of obtaining Vi, V2, V:^ does not require knowing 
Ui, U2, u^. Of course if one decided to separate v^ and v-j 
first, one would need to know these last three vectors but not 
We and u^. 

In connection with orthogonalizing to known eigenvectors 
we remark that if x is already orthogonal to such vectors 
then in theory x' and all successive approximations will be. 
In practice however the orthogonality is lost by round-off 
and nuist be regularly restored by direct calculation. 

The preceding method was, in the main, the one used in 
the computation. However, there is a variation of the pro- 
cedure that is of interest. It takes advantage of the fact 
that we may make the iteration scheme move upward or 
downward on the scale of eigenvalues and enables us to 
reduce the number of orthogonalizations. Consider the 
problem of finding v\, Vo, v^. We first apply the iteration 
procedure that increases //(a;), that is, x' =x-\-a^, with an a 
apj)ropriate to the whole space. After a certain number of 
steps we have eliminated the lower eigenvectors and are 
operating in the invariant 3-space of Vi, V2, v-^. We now in- 
crease a to a value corresponding to the three space. In this 
way we separate out V\. In order to avoid the introduction 
of higher eigenvectors through round-off, we intersperse use 
of the larger value of a with use of the older smaller value 
(this replaces the orthogonalization to v^^ v^^ u^, U7, us of the 
preceding method). The next vector V2 is obtained in the 
same way, maintaining orthogonality . to Vi. The vector V3 
is found by orthogonalizing to Vi and V2. To apply this 
techni(|ue to Vty and Vj we first find v^ and then use the a 
iteration with decreasing fx, that is, x'=x — a^. Then vj is 
found by orthogonalizing only to v^, and vq by orthogonalizing 
to Vh and v-j. 

If we analyze either of the above methods of separation 
in the way we earlier compared the fixed a method with the 
power method, we find again that in the later stages of the 
iteration we are applying the power method. We first elimi- 
nate all but the invariant subspace corresponding to the close 
eigenvalues, and then, in essence we use the |)()W('r method on 
a linear combination of A and / that will separate out the 
desired vectors. Thus, this method is closely related to 
that explained at the end of the text. 

The final eigenvalues were found with a relative error of 
10~^. The absolute in the components of each eigenvector 
were determined with an absolute error of 10*^, when the 
largest component of each vector is taken to be 1. 

Los Angeles, September 28, 1950. 
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